Despite the strong research advances in computer vision and pattern recognition of the last decades, marker-less vision-based applications are rare in industrial environments. This is mainly due to the expensive engineering step needed for their integration into an existing industrial workflow. Typically, an expert decides which algorithm is most suited for each specific application. The decision is generally based on not only the geometry and the appearance of the object to be recognized by the application, but also the illumination conditions and the optical sensor (the camera and the lens) that are used.
For example, when the object is piecewise planar and textured, the expert may select methods based on feature point detection and/or template-based tracking to recognize the object in a single image or an image sequence. He manually tests different visual feature detectors and descriptors, and then selects the method that provides the optimum recognition results for the object. When tracking should also be performed, he typically also needs to choose the planar polygons in the CAD model to consider in the template-based tracking. The problem is that the result of such an engineering step typically cannot be used for other objects, for example, if the industrial object is not piecewise planar and not well textured.
Consequently, most of the engineering steps are not general and cannot be easily adapted to new applications. In fact, they are often only valid for limited object categories, restricted viewpoints (even within the same category of objects), special illumination conditions or camera resolutions and lenses. Therefore, in practice, once one of these parameters changes, the engineering step must be performed again.
There are already many computer vision methods available that are working on 2D structures and perform a feature matching. One such method is the “scale-invariant feature transform” (SIFT) detector and descriptor [1]. A version modified towards speed is the “speeded up robust features” (SURF) method [2]. Certain other descriptors based on classification were also published, like randomized trees [3], randomized ferns [4], and a boosting method [5]. Closer to the concept of agents, which is explained below, are the feature detectors like the “Harris affine” or “Hessian affine” detectors [8], “intensity-based regions” (IBR) [9], “edge-based regions” (EBR) [9], “maximally stable extremal regions” (MSER) [10], “salient regions” [11] and others, which where summarized and evaluated in [6]. Although they don't provide matching methods, they give an approximate transformation as soon as a matching has been established. Another type of features is edges. Edge features are more difficult to handle, but also some methods have been published [7], [11]. Contrary to the approach of the present invention, all these algorithms are not incorporated into CAD models and don't make use of the additionally available geometric data.